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ABSTRACT 

The regression model Y = + £» with e/— N(0,o^^), has 

been studied extensively. That is, the model in which the 
errors are independent and identically distributed as 
N( O, has been studied already. 

In this thesis we study the model in which the sample 
observations are correlated with a prescribed correlation 
structure and show that many of the results available for 
the independent case apply equally well for the correlated 
samples . 

We shall find that some results obtained here are not 
just the same as the case where the errors are independent 
and identically distributed as N( 0^0"^), 
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I . INTRODUCTION 



Regression analysis is designed for situations where a 
variable is thought to be related to one or more other 
measurements made, usually, on the same objects. A pur- 
pose of the analysis is to use data (observed values of 
the variables) to estimate the form of this relationship. 

The problem we consider is that of estimating or pre- 
dicting the value of a dependent random variable y on the 
basis of some known measurements of an independent controlled 
variable x. Scientists, economists, psychologists, and 
sociologists have always been concerned with the problems 
of prediction. Meterologists are constantly analyzing 
data in hopes of predicting or forecasting with a high 
degree of accuracy. 

An example would be to use information on weight and 
height of people to estimate the extent to which a man's 
weight is related to his height. If among people one was 
picked at random v/e might expect his weight to be a + bx 
(where x is height), namely, E(y) = a + bx (where y is 
weight). In gathering data, the weight of every man with 
height X will not be exactly a + bx. Therefore the differ- 
ence, yi - E(yi) , can be written as 

ei = yi - E(yi) = y^ - a - bx| , 

hence yi = a + bxi + ei . 
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The man's age can also be considered to be a factor affecting 
weight. Then the model is extended to be 
• E(y) = a + b-|X-| + '^2^2 * 

More generally, we can extend the equation to be 
E(y) = a + b-|X-| + ^ 2^2 + • • •+ bj^xj^ 

yi = a + b-jx^ + b 2 X 2 + ' ' * + bj^Xj^ + . 



A frequently used notation is 

^ *>2='l2 + ■ ■ ■ + + e. , 

for i = ,n, with x^q = 1 for all i. 

Now define the following matrices and vectors: 



X 10 1 ... X.,J^ 




' 1 Xi, ... 


^20 ^21 • ' ‘ ^2k 

« « e 


11 


1 Xj, ... x^^ 

' » 


* ♦ • 
• m 




* • ^ 

, 


X ^ X . .... X , 

nO n1 nk j 




lx. ... X , 

1 n^ nk J 





f N 

'’0 








r 


B = 




Y = 


^2 


e = 


®2 




.*’k. 


, , 


. "n, 


9 


e 

n j 



Then vie write 

y = 2^ + e , with E(Y) - ^ . 

The model Y=XB + e , with e~N(0,o^I), has been studied 
extensively. That is, the model in which the errors are 
independent and identically distributed as N( 0 , (T'^ ) has 
been studied already. 
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In this thesis we study the model in which the sample 
observations are correlated with a prescribed correlation 
structure and show that many of the results available for 
the independent case apply equally well for the correlated 
samples . 

In this thesis, we shall examine the model 

Y = ^ + e , with e~N(0,V) , where 

V = ?g(A + A» )+oC(I-E) , 



A 




Q. ^ 8. ^ * 

2 .2 



a a • ' • a 
L n n n J 






A* is the transpose of _A, a^( i = 1,2,-",n) and oC are posi- 
tive constants, _I_ is an n x n identity matrix, and ^ is an 
n X n matrix all of whose elements are unity. 

We shall find that some results obtained here are not 
just the same as the case where the errors are independent 
and identically distributed as N(C;0^). 
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II. SUMMARY OF KNOWN RESULTS 



A. SOME PROPERTIES OF B AND UNDER NORMAL THEORY 

The model Y = O + e, e'^N(0, I), where the components 

of B and o-^ are unknown, and X is a known matrix of rank 
k + 1. Since the vector of errors e is normally distributed, 
the maximum-likelihood method will be used to estimate B 
and cr^. Solving the likelihood equation we get 

= S“^X'Y , where S’^= ( X'X )“^ , 

"""a 2. (I-M)'(y-2^) _y»(i - 2^"^x» ) y 

n-(k + 1) n-(k + 1) 

^ A 2 

Then, B and have . the following properties: 

1. consistent 

2. efficient 

3. unbiased 

4. sufficient 

5. I -- N( P , S"^ ) 

6. complete 

7. minimum variance unbiased 

8. ( n - k - 1 ) 5^ 1 V ( n . k - 1 ) 

9. B and (p- are independent. 
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B. SOME THEOREMS USED IN DERIVING THE RESULTS 



Theorem 1. If Y is distributed as N(/U.<r?I), then 

- — j 

is distributed as X(k» , where X - ^ k = rank 

of A, if and only if A is idempotent. 

Theorem 2 . If Y is distributed as N(^,V), then Y’^ is 
distributed as "X^(k,X) » where X = and k = rank of 

B, if and only if ^ is idempotent. 

Theorem 3 . If Y>~N(,U,y), then Y'M and Y’^ are independent 
if and only if AVB = 0. 

Theorem 4 . If Y~N(yjV)» then C'Y and Y'AY are independent 
if and only if C'VA = 0, 

Theorem 5 . (Hogg and Craig theorem) 

Let Q = Q-J+Q2 ■*■■■■ » where Q>Q-j» ••• »Qj^_-|and 0 .^ are k + 1 

random variables that are quadratic forms in the observa- 
tions of a random sample of size n from NCiCjCh"). 

Let -3- be 9C*'0T) i let-— be ^ ' ' ' »k-1 , and 

let be non-negative. Then the random variables Q^, Qg, 
***’ *^k mutually stochastically independent and, hence, 

^ i ■= 1 ^ 

Theorem 6 . (Baldessari theorem) 

Let Y be N(d, V ) and Bq, B^, * ’ ’ , Bj^ be ( n x n) idempotent 
matrices satisfying 

^ B . = X - E , where I and E are matrices as 
j.o 

defined above. 

Let be a positive constant. 
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Then a necessary and sufficient condition for » 

j = • ,k , to be mutually independent and have 

non-central Chi-square distributions with r . (r .=rank of B . 

J J J » 

j = 1,2f' • ‘ ,k) degrees of freedom is that the covariance 
matrix V has the following structure 

V = 3g(A + A» )+c<(I-E) . 
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III. SOME PROPERTIES OF B AND I'(1-M"^X')Y 
UNDER THE PRESENT MODEL 



We consider the model 

y = XB + e , e-N{0,V), where V=^(A+A’ )+ ^ (I-E) . 



Let B=S-’x'Y and Y'CY _ (Y-m) ’ (Y-M) Y'(I-^"X')Y 



.-1 



Now we will show that 

A. E(|) = B 

B. Y‘CY ^2, , , , 

X ( n - k - 1 ) 

C. E(Y'^)= cA (h-k-1 ) and 

D. B and Y'CY are not independent. 

A. EXPECTED VALUE OF B 

E(B) = E(S"^X»Y) = S"''x'E(y) = S"^X'E(2^ + e ) 

= B 

^ -1 

Hence B = S X'Y is an unbiased estimate of B 



B. DISTRIBUTION OF ( I ~ 2^ ) * ( I - ) 



n-iC- I 



We have seen that (n-k-1 ) ^^(n-k-1 ) , where 

V • f T y ^ y * ^ V 

Therefore we can conjecture that — — 



is distrib- 



'X 



uted as ^^"(n-k-l ) . The necessary and sufficient condition 

’to be ^“(n-k-1 ) is that — ^ is idempotent. 

Now we want to show that ^ ^ )]iis idempotent, 

that is, to show that -L ( I-^"^ X' 2S’ ) = I' • 
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Note 



= i )(^(A+A« )+ (I-E)) (l-2iS~^X') 

= (i-2^"^x’) , 
since 2^"^X'E=E , 2^“^X»A’=A» and AXS~^X’=A . 

Hence, by theorem 2, X'£X Q > A ) 

C< * 



where q = rank ( I - ) 

.1 

= rank ( ^ - 2^ X* ^ 

= tr ( I - XS“^X' ) 

= tr ( I ) - tr ( X'2^"' ) 
= n - k - 1 



,-1 



since I-XS ^X' is itself symmetric and idempotent, 

-1 

and ^ ( I - ^ ^ = o . 



Therefore 



Y»£f 

cC 



( n-k-1 ) . 



C. EXPECTED VALUE OF Y'CY 



A 

From B it is obtained directly that E(y’ (X“2^ X' )X)= 

vpY 

c4(n-k-1 ) also then — is an unbiased estimate ofoc. 

n-k-1 

D. B = S"^X’Y and Y'CY are NOT INDEPENDENT 



To show that they are not independent, let's show 
S"^X»VC = Y^2S'X^i”21S'’^X' ) 0 ( Th. 1| ) . 

X'V(I-^”^X' )=S"^2i’ ('s(A+A» )+ -^CI-E)] (I-}^'’''x» ) 
=^'g(S"^X'A»-S'’^X*A'XS”^X» ) =V 0 . 



A 

We recognize that B and Y'CY are not independent, which 
is not the same results as obtained in the i.i.d. case. 
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For reference, the variance and covariance of B is 
provided as follov/s: 



V(B)=V| 






- 1 - 



a-o((n-1 )/n+(nax’-a*X^ +ix* )S x 
Sj"* (5§Xja-^nax-^x) 



(^a *X^ -hnax' -o(x ’ ) 



-1 



•n 



where a = (1/n)22 a. , a’ = ( a* a„ 

i-1 ^ ^ 






% ^ * 



2^1 = 



and 



( X.^ 


X • 2 * ' ' ^ 


' ^.11 


... X. , 

1 k 
1 
f 




P 

' • ^ nk 



*k X. .=(1/n)^x^ . for all j , 



i = l 






Note that <X factors from the lower right-hand corner, 
but not from the rest of the matrix. This causes trouble 
because the unknown parameter <J- does not cancel out in 
forming the usual Student t statistic. 

Therefore we develop the new modified model without 
Bq in the next chapter. 
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IV. SOME PROPERTIES OF AND Y*’( I - x'^’ ) y'' 

UNDER THE NEW MODEL 



We consider the model 



T = + 1 



where y'"' = Y - yl = (I - (1/n)E )Y 

X^-(x.^lj • ^ = ( I 



X*^ = 



(1/n)E )X^ 






( b, bj 



\ >' 



e = e 



- e1 = ( I - (1/n)E )e 






Note then that y? = y^-y , thus 



T» 



it y = zi x:.=o, that is, i»y =o, i«x = o 

i=( ^ kH 



Then 



V(e*"') = ( I - (1/n)E ) , 

i.e., e'"'- N( 0 , <(I-(1/n)E ) ) 



and 



N( X'^'b^ , 



-X ( I - (1/n)E ) ) 



under our original assumptions. 

Let = S””*'x'''y“ , where S“ = X“ X'' , and 



<-K 



Y'X- > ^ Y* ’ ( I - X'" s'"' “ ^ X”* ' ) Y'”* _ ( Y''‘'-X'"'B., ) » ( y’''’-X'"'b^ ) 






We are going to shov/ 

A. The expected value of 

-X- * •Jf 

B. The distribution of X £ X 

-X 

C. The expected value of X” £ X 

-x- 

D. That B^ and Y C Y are independent. 
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A. EXPECTED VALUE OF 

E(B^) = E(s'''‘"^X‘^'y*) 

-!5.f 



= S'^-'X E(rB,+e'') = 



Hence B^=S 'x Y 



is an unbiased estimate of B. it is 

ys J- 



easily shown that B^ is the same as B^.in £ 



B = Po' 






B . DIS TR I BUTI ON OF ( 1 /c< ) ( I"' * C'Y' ) 

Since C" ( I-^E) =I-X'V*”^ - 7 ^ I 

(1/oc)Y'”*’c‘VLX^(q,X) by Th. 2, where 
q = rank (1 /ot ) (^-X''S" "'x” ) 

= rank ( I-xV'"^ X’"' ' ) 

J‘i. “1 .V. f 

= tr (I-X'^S*' X” ) 

= tr (I) - tr (X’*' X”S'‘"M 



is idempotent, 



= n - k 






and X = hBJl x" ' ( 1 A ) ( I-X" ^ X*"’ * ) ^ 



Therefore 



( 1 /x) (Y*"*'cV'') ^ % ( n - k ) 



Y C Y 



-Yi-. 



C. EXPECTED VALUE OF 

It is obtained directly from B that E(Y £ Y )=c/Un-k), 

v*‘* ^ 

Also it is easily seen that — „ .-jr is an unbiased estimate 



n-K 



of . 
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D. B^=S X Y 



AND Y^'’c'V” ARE INDEPENDENT 



V*. i V, f -*»- -V. ^ 1 ^ I 

S" ^X" oc(I-(1/n)E)(I-rs” X ) 

= o((S” ^X“ -d/n)S'' ^X“ E)(I-X”S ^X” ) 



= 0 



t 



( note X” E = 0 ) 



Therefore, hy Th. 4, they are independent. 



1 ? 




:= « 

0 ^ 







V. INTERVAL ESTir/IATION AND PREDICTION INTERVAL 



The construction of interval estimators and a prediction 
interval to contain a single future observation in the i.i.d. 
case with a regression model has been discussed in many- 
standard texts. In this chapter we extend these results 
to the case where the samples are correlated with a special 
type of covariance structure. We shall discuss 

A 

A. Covariance of ) 

B. Interval estimation of r'B^ 

C. Prediction interval for one future observation. 



A. COVARIANCE OF B^ , V(B^ ) 

Since is equal to the product of a constant matrix 
S X and a normally distributed vector Y*, B^ has the 
k-variate normal distribution. 






We have already shown that the mean of B^^ is B^ . 

The covariance matrix of B^ is 
V(B^ )=E((B^ -B^ ) (B^ -B^ ) ») 

=E( (s'“'“^x*'''’r’*-B^ ) (S’“’"^X” ) ’ ) 

=E( (r-'r ’ (X%+e-")-B^ ) (S-"“'x"*' (X%-^e-")-B^ ) • ) 
=S'- V(e'“)X ^ 

= ^s” (i-(i/n)E)rs^' ^ 



= d s 



-x-1 



So B^ is distributed N( B^ , <i. S'””'* ) 



18 



1 ?* 

VlOiTli 



A 

ud .A 

- i 

f •#- 

i » 

•#■ ^ 



■?l 



i. 



B. INTERVAL ESTIMATION OF r'B^ 



Frequently, an experimenter is interested in setting 
confidence limits on some function of . A method is 
available for one of the most frequently occurring cases, 
i.e., for the case of a linear function of the b^ . 

We, here, discuss the case of a linear function of the b^ 
(except b^ ). Let r be a known k x 1 vector of constants, 
then, to set confidence limits of size on rB^ ,v/e 

proceed as follows. 

Since r'B^ is distributed N (r *B^ ,xr • S'*”^ r ), it is 
clear that is distributed N(0,1). We also 

know Y** ^ (n-k) and independent of . 



Therefore, £'B-,-r»B^ / (n-k) 



(oC 



r'S 






\ i- 
r )'5 



i- y „ ' 

y'“' c'"'y' 



t ( n - k ) 



We arrive at the probability equation 



P(r'B^ 

where 



-ct^ £ ) 



c = 



-''.I j'- 

(r'S'' r)Y'' C‘'Y' 



n-k 



cA\ , 



C. PREDICTION INTERVAL FOR ONE FUTURE OBSERVATION 



Hahn /~4_7 has derived prediction intervals to contain 
some future observations of i.i.do random variables from a 
normal distribution for a regression model. In this section 
we extend this result to the case v/here the samples are 
correlated and have a special type of covariance structure 
to construct prediction interval to contain a single future 
observation. 
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Let be the one future sample. 

Define Y to he the (n+1) component vector including the 
original sample of size n and • We assume Y^N (XB, V), 

where X is (n+l)x(k+l), of rank k+1 and V is as defined in 
III, and is (n+l)x(n+l). 

Define y*+]^ as follows: 

yn+1=yn+1-5n.1 = (1 /(n-M ) ) E' y . . 

Then, y*^1=y„^.,-(l/(n+1))(y^^,+ny^) = (n/(n+1 ) ) (y^^, -5„) , 

where y =(1/n)£;yi 

n l=l ^ * 



and y* 4 .j^ can he expressed 

^n+1"^^/ ^ ^ ■‘■^2^(n+1)2'*’ ’ ‘ ’ ‘‘'^k^n+D k'*’®n+1 ^ 

= (n/(n+1))( ) , 



where X'”\,,= ( x’' 



-n+1 “ " ""n+1 1 » ^n+1 2 * * " » ""n+1 k 

•X* V’ 

We also, therefore, know E(y^^,^ )=n/in+lJ and 

V(y*+-, )= in/n+1l)< since 

V(y'"+i ) = (n/(n +1 ) )^(V(yn+i )+V(yn)-2C0V(y^^^ ,y^) ) 

= (n/ (n+1 ) +a-( (n-1 )/n)'^-2(^a+'sa^_j_^ - -<) ) 

= (n/(n+1))^ . 



f V ) 



Next, let y”^.-| =(n/(n+1 ) )B,[X“^ 



then E(y"^,, ) = (n/(n+1 ) and 

V(y*,H )= ^ (n/(n+1 ) 
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Now let z = y*^^ , 

then E(z) = 0 

V(z) = c^(n/(n+1 ))(1+(n/(n+1 )) , 

Therefore, z^-NC 0 , c< (n/n+D) (1 +(n/n+1))X^^-l S” , 



The standardized variate z' = z/(V(z))'^ is normally 

distributed with mean 0 and variance 1. Furthermore, 
since (1 At )Y^ ' C''Y”--9C^(n-k) and is independent of z * , 



the ratio 



T=- 



i'r t -Yr -Y- \M 



'Y C Y 
> oc(n-k) 



\'2 



A.H. . 

^^n+1 "^n+1 I 



n - k 






|(n/n+l ) + (n/n+1 S'-'' 



W I 

"i' ' VT *\ ) 



follows a Student t distribution with n-k degree of freedom. 

* • * * 

Proof of independent relationship between Y C Y and z. 
Y C Y and z = can be expressed in terms of 



I = (y^. ‘‘‘» Y^+i)' as follows: 

’ c*V"= V . |l- ( ) E 

^=yn+1 -yn+1 

= (n/n+1 ) ( ( --i , • • • - 1 ) S" X'"' ^ I- ( 

=(n4n+D)BY 



C (I-(1/n)E;0)Y = Y»D’C’'*DY 



Then it can be seen that BVD'C D = 0, where V=h{A+A') 

+ a(I-E) is an (n+l)x(n+l) matrix. 

Therefore, bY Th. 4, they are independent. 

Consequently, a two sided lOOr^ prediction interval to 

* 

contain the future sample Yj^+]_ using the regression obtained 
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■S " y 



«tb 

'o!m 



• .-ilot 



» =1 







from the n given observations is 



^n+1 



( (n/n+1)) + (n/(n+1))2x'~'|^ ) (Y^'’c'V') 

n - k 



Finally, the lov/er limit, L, and the upper limit, U 
are expressed as follows: 

L = ( (n+1 )/n) - t r u ) + y^ 



U = ( (n+1 )/n) (y‘"'_j_^ + t r u ) + y^^ 

z 



where u 



^ (((n/tn+1)) + (n/(n+1))r|^ S'""^X^^^ ) (y'"‘ ’( 

^ 



h 



, for 
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